While the NLP community has produced numerous summarization benchmarks, none provide the rich annotations required to simultaneously address many important problems related to control and reliability. We introduce a Wikipedia-derived benchmark, complemented by a rich set of crowd-sourced annotations, that supports $8$ interrelated tasks: (i) extractive summarization; (ii) abstractive summarization; (iii) topic-based summarization; (iv) compressing selected sentences into a one-line summary; (v) surfacing evidence for a summary sentence; (vi) predicting the factual accuracy of a summary sentence; (vii) identifying unsubstantiated spans in a summary sentence; (viii) correcting factual errors in summaries. We compare various methods on this benchmark and discover that on multiple tasks, moderately-sized fine-tuned models consistently outperform much larger few-shot prompted language models. For factuality-related tasks, we also evaluate existing heuristics to create training data and find that training on them results in worse performance than training on $20\times$ less human-labeled data. Our articles draw from $6$ domains, facilitating cross-domain analysis. On some tasks, the amount of training data matters more than the domain where it comes from, while for other tasks training specifically on data from the target domain, even if limited, is more beneficial.

In this blog, I have tried summarizing the paper Multi-Sentence Compression: Finding Shortest Paths in Word Graphs as per my understanding. Please feel free to comment your thoughts on the same! Sentence compression is the task of compressing a long sentence into a shorter one by deleting redundant words. This paper proposes a novel unsupervised multi-sentence compression technique wherein the goal is to represent a set of related sentences by a single sentence in such a way that it preserves the important parts of the main content and is also grammatically correct at the same time. One of the possible use cases for Sentence Compression (SC) is in Extractive Text Summarization, wherein a typical flow is to rank sentences based on a certain scoring strategy followed by selecting top-k sentences as representative summary.